Audio Description of Art : The Role of Mental Imagery and Embodiment

How can we make visual art accessible for audiences with visual impairment and blindness? Can audio description help blind and visually impaired (BVI) audiences to understand and experience art? Which ingredients must AD contain to evoke and stimulate the creation of vivid internal mental images? Can AD engage the target audiences and contribute to their aesthetic experience and enjoyment? What other means are there to enhance inclusion and immersion? The paper focuses on how audio description can evoke and stimulate the creation of vivid internal images via verbal descriptions of art and thereby contribute to an embodied aesthetic experience for the BVI audiences. It introduces research on image perception, image description, and mental imagery, relevant for audio description; summarise guidelines and recommendations for audio description of art; and offers authentic examples of AD of art tailored for the target audiences. The paper underlines the importance of reception studies and discusses the issue of inclusion and multi-sensory experiences

Uncovering Scientific and Multimodal Literacy through Audio Description

Today’s scientific texts are complex and multimodal. Due to new technology, the number of images is increasing, as is their diversity and complexity. Interaction with complex texts and visualisations becomes a challenge. How can we help readers and learners achieve multimodal literacy? We use data from the audio description of a popular scientific journal and think-aloud protocols to uncover knowledge and competences necessary for reading and understanding multimodal scientific texts. Four issues of the printed journal were analysed. The aural version of the journal was compared with the printed version to show how the semiotic interplay has been presented for the users. Additional meaning-making activities have been identified from the think-aloud protocol. As a result, we could reveal how the audio describer combined the contents of the available resources, made judgements about relevant information, determined ways of verbalising visual information, used conceptual knowledge, filled in the gaps missing in the interplay of the resources, and reordered information for optimal flow and understanding. We argue that the meaning-making activities identified through audio description and think-aloud protocols can be incorporated into instruction in educational contexts and can thereby improve readers’ competencies for reading and understanding multimodal scientific texts

Syntolkning som multimodal medierande aktivitet

Audio description (AD) offers people with visual impairments and blindness a richer and more detailed understanding and experience of film, theatre and other events by describing what is seen but not heard (Holsanova 2016b, 2020a). With the analysis of audio described film scenes, I will illustrate that AD is a complex meaning-making activity that cannot be limited to only replacing the visual part with a verbal part. Audio describers are aware of the interplay between the different modes of expression, create content links and take into account the meaning of the film sounds, music and dialogue when deciding what needs to be described from the visual scene, how it should be described and when. The goal of AD is to complement what is missing in the multimodal interplay in order to achieve a comparable understanding and experience of the non-sighted audience (Reviers 2017). Audio describers are thus engaged in a complex multimodal mediating activity (Braun 2007, Holsanova 2020a, 2021)

How the blind audience receive and experience audio descriptions of visual events – a project presentation

We present a three-year multidisciplinary research project that started in 2019 and is funded by FORTE (Swedish Research Council for Health, Working Life and Welfare). The theoretical aim of the project is to gain a better understanding of the principles that underlie successful communication between the sighted and the blind via audio description (AD). The aim of a series of experiments is to identify similarities and differences in how the sighted and the blind understand, segment and experience visual, spatial as well as temporal properties of an event. The applied goal is to increase the quality of AD and to support the training of audio de-scribers and AD practices, and ultimately facilitate the understanding and accessibility of visual information for the visually impaired

Communication, cognition, and technology

Udostępnienie publikacji Wydawnictwa Uniwersytetu Łódzkiego finansowane w ramach projektu „Doskonałość naukowa kluczem do doskonałości kształcenia”. Projekt realizowany jest ze środków Europejskiego Funduszu Społecznego w ramach Programu Operacyjnego Wiedza Edukacja Rozwój; nr umowy: POWER.03.05.00-00-Z092/17-00

Event boundary perception among the visually impaired in audio described films

Audio description (AD) serves a critical role in making film narratives accessible to visually impaired audiences, aiming to enhance their viewing experience and comprehension. One method to assess the comprehension of film narratives, is through an event segmentation task, wherein participants delineate the narrative unfolding into distinct meaningful events. In the present study, both sighted and visually impaired participants engaged in such tasks. Sighted participants watched a Swedish film, while visually impaired participants experienced the same film with two AD versions—one explicitly expressing key event boundaries and another containing more implicitly conveyed ones. Our findings indicate that visually impaired participants perceived event boundaries similarly to sighted participants, suggesting that AD effectively conveys the event structure. However, in the AD version with implicit expressions, event boundaries were less likely to be recognized. These results shed light on event segmentation dynamics in films, emphasizing the importance of how event boundaries are presented in AD. This has significant implications for improving the cinematic experience for visually impaired viewers, emphasizing the need for clear, explicit information about event boundaries within AD

To tell and to show: The interplay of language and visualizations in communication

Udostępnienie publikacji Wydawnictwa Uniwersytetu Łódzkiego finansowane w ramach projektu „Doskonałość naukowa kluczem do doskonałości kształcenia”. Projekt realizowany jest ze środków Europejskiego Funduszu Społecznego w ramach Programu Operacyjnego Wiedza Edukacja Rozwój; nr umowy: POWER.03.05.00-00-Z092/17-00

Meaning-making made aloud : From a printed version to an audio version of a multimodal popular science journal

First, the printed journal is analysed in accordance with Unsworth (1997), focusing on how the resources of text, images and graphics are deployed in scientific explanation and how the meaning is constructed by the visuals. Second, the interpretative process of meaning-making is uncovered by think aloud protocols. In order to produce an aural version of the complex text, the interpreter must assess what to describe, how to describe it, and when to describe it (Holsanova 2015). The audio describer combines the contents of the available resources, makes judgements about relevant information, ways of verbalizing it, fills in the gaps missing in the interplay of the resources and re-arranges the order of information for optimal flow and understanding. In this way, he contributes to multimodal literacy (Walsh 2010, Kress & Jewitt 2003). The aural version of the journal is finally compared to the printed version to show how the semiotic interplay has been realized for the end users

Making popular science accessible: Uncovering AD skills and competences

The paper focuses on skills and competences used in audio description (AD) of a Swedish popular scientific journal. The journal, containing complex images and visualizations, is made accessible for visually impaired audiences by producing an audio version. The task is to create a verbal description to enhance understanding and enjoyment. The interpretative process of meaning-making is uncovered by think-aloud protocols during the task. The protocols are transcribed, coded and analyzed. The aural version is compared to the printed version to show how the content has been realized for the end users. The skills and competences of the audio describer are revealed through a combination of both, the think-aloud protocols and the comparison of the printed and audio versions of the journal. In result, AD skills in this context include: (a) Selecting skills. Judgements about relevant information.(b) Expert knowledge of the subject area. (c) Knowledge of various types of images and graphics. (d) Language skills. Various ways of verbalizing visual content.(e) Integration skills. Combining the contents of the available resources, creating semantic links, filling in the gaps in the interplay. (c) Interpretation skills. Using conceptual knowledge for interpreting schematic images. (d) Rhetorical skills. Choosing entry points and reading paths. Re-arranging the order of information for optimal flow and understanding. (e) Pedagogical skills. Grouping similar information. Creating summaries and introductions to modules. (f) Facilitating understanding and cognitive processing. Considering working memory capacity of the recipients. Conveying information in ’digestable’ portions. Making the description short and comprehensive. Repeating information for better understanding.(g) Vocal skills. Using voice, intonation, speech rate, emphasis and pauses to highlight and group information. (h) Technical skills (studio recording and editing)

Med blick på nätnyheter. Ögonrörelsestudier av läsning i nätbaserade tidningar. (Looking at the net news. Eye tracking study of net paper reading)

How do readers interact with net papers? Does the new medium influence our way of searching for news? Does net-paper reading differ from newspaper reading? 12 users were asked to read 2 net-papers at their own pace. The aim of the study was partly to analyse reading behaviour on the Internet, partly to understand some of the underlying rationale and motivation of the behaviour. To achieve this aim, we used a combination of three methods: (i) eye tracking, (ii) retrospective verbal protocols supported by the replay of the interaction and (iii) interview data. In our analysis, we focus both on the readers' actual interaction with net papers (their reading paths, entry points, scanning and reading activities, orienting and navigating) and on their reflections, experiences, comments and attitudes towards the new medium